Food inspection assisting system, food inspection assisting apparatus and computer program

ABSTRACT

The present embodiment provides a food inspection assisting system, a food inspection assisting apparatus and a computer program for accurately and rapidly inspecting food. According to one embodiment, a food inspection assisting system includes a carrying apparatus, a sensing apparatus, an assisting information generator, an information output apparatus and a controller. The carrying apparatus carries a food item. The sensing apparatus senses the food item carried. The assisting information generator generates inspection assisting information for the food item based on output information of the sensing apparatus. The information output apparatus that displays the inspection assisting information. The controller controls timing to display the inspection assisting information based on time required to carry the food item to an inspection area in which the carried food item is inspected.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2018-015921, filed on Jan. 31,2018, the entire contents of which are incorporated herein by reference.

FIELD

Embodiments of the present invention relate to a food inspectionassisting system, a food inspection assisting apparatus and a computerprogram.

BACKGROUND

In the process of food production, removal of foreign matter and defectsfrom raw materials is performed in an operation of screening the rawmaterials before being processed. For example, in the production ofprocessed foods containing chicken, removal of foreign matter such asbones and feathers and defects such as blood-containing part isperformed in the operation of screening the raw materials. It is desiredto reduce operational errors and speed up the screening operation.However, increasing the number of workers incurs high production cost.

In the process of food production, introduction of apparatuses forassisting accurate inspection, such as an X-ray inspection apparatus,has been proceeding. In the screening operation using the X-rayinspection apparatus, if, for example, the mixing of a bone in a rawmaterial is found in an X-ray image, a worker compares the X-ray imageand the raw material while searching for the bone by visual check orpalpation, and removes the bone found. However, this has problems inefficiency and accuracy, such as that it may take time to find theposition of the bone and that the bone may be overlooked. Besides bones,feathers and blood-containing part also need to be removed, and alsoneed to be detected accurately and efficiently.

PRIOR ART DOCUMENT

[Patent Document 1] Japanese Patent Publication NO. 2017-142133

[Patent Document 2] Japanese Patent Publication NO. 2007-286041

[Patent Document 3] Japanese Patent Publication NO. 2005-233636

Problem to be Solved

The present embodiment provides a food inspection assisting system, afood inspection assisting apparatus and a computer program foraccurately and rapidly inspecting food.

SUMMARY

According to one embodiment, a food inspection assisting system includesa carrying apparatus, a sensing apparatus, an assisting informationgenerator, an information output apparatus and a controller. Thecarrying apparatus carries a food item. The sensing apparatus senses thefood item carried. The assisting information generator generatesinspection assisting information for the food item based on outputinformation of the sensing apparatus. The information output apparatusthat displays the inspection assisting information. The controllercontrols timing to display the inspection assisting information based ontime required to carry the food item to an inspection area in which thecarried food item is inspected.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of a food inspection assisting system accordingto a first embodiment.

FIG. 2 is a cross-sectional view of an image capturing apparatusaccording to the first embodiment.

FIGS. 3A to 3C show an example of an image obtained without colorfilters, an image obtained through a blue filter and an image obtainedthrough a green filter.

FIGS. 4A and 4B show an example of a binary image and inspectionassisting information.

FIGS. 5A and 5B show a specific example of the system according to thefirst embodiment.

FIG. 6 is a flowchart of an example of operations according to the firstembodiment.

FIG. 7 shows an example of a captured image of chicken according to asecond embodiment.

FIG. 8 is a cross-sectional view of an image capturing apparatusaccording to the second embodiment.

FIG. 9 shows a specific example of image processing according to thesecond embodiment.

FIG. 10 shows an example of an image captured by a camera according to amodified example of the second embodiment.

FIG. 11 shows an example of respective binary images according to thesecond embodiment and the modified example thereof.

FIG. 12 shows an example of a food inspection assisting system accordingto a third embodiment.

FIGS. 13A and 13B illustrate continuous image capture performed by theimage capturing apparatus and continuous projection performed by aprojector.

FIGS. 14A to 14C show an example of captured images obtained bycontinuous image capture.

FIGS. 15A to 15C show an example of a plurality of pieces of inspectionassisting information continuously projected.

FIG. 16 is a flowchart of an example of operations according to thethird embodiment.

FIG. 17 shows an example of a food inspection assisting system accordingto a fourth embodiment.

FIG. 18 shows projection performed by a projector.

FIG. 19 is a flowchart of an example of operations according to thefourth embodiment.

DETAILED DESCRIPTION

The following describes embodiments of the present invention withreference to the drawings. In the drawings, the same components areprovided with the same reference numerals, and the descriptions thereofmay be omitted.

First Embodiment

FIG. 1 shows a configuration example of a food inspection assistingsystem (hereinafter “system”) according to a first embodiment. Thesystem of FIG. 1 includes a food inspection assisting apparatus(hereinafter “inspection assisting apparatus”) 1, an image capturingapparatus 2 that is a sensing apparatus, a display 3 that is aninformation output apparatus, and a carrying apparatus 4.

In the system of FIG. 1 , a food item that is to be inspected (targetfood item) is carried by the carrying apparatus 4 in the right directionalong the paper plane at a constant speed. The side from which the fooditem is carried is referred to as the upstream side, and the side towardwhich the food item is carried is referred to as the downstream side. Onthe upstream side, food items are placed on the carrying apparatus 4manually by a worker or by a machine at constant time intervals or atarbitrary time intervals. When a sensor 23 detects that a food itementers an image-capturing region in the housing of the image capturingapparatus 2, a camera 32 captures an image of the food item. The figureillustrates a situation in which an image of a food item 22 is captured.For each food item, the number of times of image capture may be one, ora plurality of times of continuous image capture may be performed (forexample, ten times for one second). In the case of continuous imagecapture, the food item moves in the carrying channel even during theimage capture, so that captured images are obtained in which theposition of the food item gradually changes in the image-capturingregion.

The captured images as output information of the image capturingapparatus 2 are sent to the inspection assisting apparatus 1, and theinspection assisting apparatus 1 performs processing of detectingforeign matter or a defect based on the captured images. Specifically,after performing various types of image processing on the capturedimages, processing of detecting foreign matter or a defect is performedbased on the processed image. Note that a food item containing foreignmatter or a defect may be referred to as a defective item, and a fooditem containing no foreign matter or defect may be referred to as aquality item.

Examples of the food item to be inspected include chicken, pork, beef,mutton, seafoods, vegetables, fruits, grains, beverages and seasonings,and the type of the food item is not particularly limited. In cases ofmeat such as chicken, pork, beef and seafood, the object to be inspectedmay be unheated raw meat or may be heated meat. The food item may be anunprocessed raw material, may be a raw cut of meat, or may have beensubjected to some processing. That is, the degree to which the food itemhas been processed is also not particularly limited.

The first embodiment is described using an example in which a raw cut ofchicken is used as an object to be inspected and a feather is to bedetected as foreign matter. The object to be detected as foreign mattermay be other than a feather (for example, a bone, a cartilage, a woodpiece, feedstuff and the like).

When foreign matter (a feather) is detected in an image, the systemidentifies the position of the feather, and generates inspectionassisting information including information representing the identifiedposition. An example of the information representing the identifiedposition is data indicating the identified position. In the presentembodiment, frame data (for example, rectangle frame data) enclosing thedetected feather is generated as such data. The system arranges thegenerated frame data on the captured image. In this manner, an image isgenerated in which the feather included in the captured image isenclosed by the frame data. The image is used as the inspectionassisting information. The data indicating the identified position isnot limited to frame data, and may be data of an arrow indicating theidentified position or may be other forms of data.

In the system, an inspection area in which the food item is inspected isprovided on the downstream side relative to the image capturingapparatus 2 along the carrying channel of the carrying apparatus 4. Adisplay 3 is arranged near the inspection area. The display 3 is anexample of an information output apparatus that displays information tothe worker. Another example of the information output apparatus is aprojection apparatus that projects information. Examples of theprojection apparatus include a projector that projects information bylight, a beam projector that projects information by light beams, andthe like. Embodiments using a projector and a beam projector will bedescribed later.

The display 3 is provided at a position such that a worker can see ascreen 3 a of the display 3 when the worker performs inspection in theinspection area. For example, if the worker is positioned on the nearside with respect to the paper plane in the figure, the display 3 isprovided on the deeper side with respect to the paper plane across thecarrying apparatus 4 such that the screen is directed to the worker.

The worker inspects the food item carried to (appearing in) theinspection area. As an example, the inspection is performed by visualcheck or palpation, or by both. The inspection assisting apparatus 1displays the inspection assisting information on the display 3 inaccordance with the timing when the food item is carried to theinspection area, that is, the timing when the food item appears in theinspection area. In other words, the inspection assisting information isnot displayed until the food item appears in the inspection area. Fromthe worker's point of view, the inspection assisting information isdisplayed on the display 3 in synchronization with the appearance of thefood item in the inspection area. In the figure, an image in whichforeign matter (a feather) is enclosed by frame data 5 is displayed onthe display 3.

The worker inspects the food item being carried while looking at thedisplayed inspection assisting information. Specifically, the workerknows the position of the feather based on the frame data 5, searchesthe portion corresponding to the frame data 5 and nearby portions forthe feather, and removes the feather found. For example, the workerdiscards the removed feather in a predetermined case. Because the fooditem gradually moves in front of the worker during the inspection, theworker needs to efficiently remove the feather. In the presentembodiment, the worker can quickly identify the position of the featherand can efficiently remove the feather by referring to the inspectionassisting information displayed on the display 3.

The system may continue displaying the inspection assisting informationwhile the food item is in the inspection area, or may display theinspection assisting information for a predetermined period of timeafter the food item enters the inspection area. If a plurality of timesof image capture (continuous image capture) of the food item isperformed, the system may continuously (for example, at constant timeintervals) display a plurality of pieces of inspection assistinginformation generated for the plurality of captured images during theperiod between when the food item enters and exits the inspection area.In this manner, the inspection assisting information can be seen as amoving image synchronized with the food item being carried. The framedata 5 included in the inspection assisting information displayed on thedisplay 3 appears to move in accordance with the movement of the fooditem. Thus, the worker can more efficiently search for the feather.

The following describes the components of the system of FIG. 1 indetail.

The carrying apparatus 4 is an apparatus that carries a food itemprovided on the upstream side by a worker, a machine or the like towardthe downstream side. Various conveyor apparatuses such as a beltconveyor, a chain conveyor, a roller conveyor, a mesh conveyor, agravity conveyor and the like can be used as the carrying apparatus 4.The conveyor apparatus is an example, and other carrying means may beused to carry the food item. Note that, while FIG. 1 shows an example inwhich the food item is carried from the left to the right, the directionof carrying the food item is not particularly limited.

At the entrance of the inspection area of the carrying apparatus 4, asensor 24 is arranged on one or both sides of the carrying channel. Thesensor 24 detects the passage of the food item. Specifically, the sensor24 detects that the food item is carried to the inspection area. Whendetecting the passage of the food item, the sensor 24 outputs adetection signal to a controller 13. Examples of the sensor 24 include alaser sensor, an infrared sensor, an ultrasonic sensor, variousphotoelectric sensors, an image sensor, a weight sensor and the like,and the type of the sensor is not particularly limited.

A collecting apparatus 4 a is arranged downstream from the inspectionarea in the carrying apparatus 4. The collecting apparatus 4 a collectsfood items that have been inspected. The carrying apparatus 4 may beelectrically connected to the food inspection assisting apparatus 1 sothat the carrying apparatus 4 is controlled from the food inspectionassisting apparatus 1. For example, the carrying speed, the turning onand off of the operation and the like may be controlled. While in thepresent embodiment, the carrying apparatus 4 carries the food item at aconstant speed, the worker may adjust the turning on and off and thespeed by operating an operating unit provided on the carrying apparatus4. Note that the food item may also be carried to the next step (e.g., acooking step) instead of being collected in the collecting apparatus 4a.

The image capturing apparatus 2 captures an image of the food itemcarried to an image-capturing region inside its housing whileilluminating the food item by a light source. Data of the image capturedby the image capturing apparatus 2 is sent to the inspection assistingapparatus 1, and is saved in an image storage 11. As an interfacebetween the image capturing apparatus 2 and the inspection assistingapparatus 1, Ethernet, wireless LAN, PCI Express, USB, UART, SPI, SDIO,serial port, Bluetooth (registered trademark) and the like can be used.These interfaces are examples, and it is not intended to prohibit theuse of other manners.

FIG. 2 is a cross-sectional view of the image capturing apparatus 2. Theimage capturing apparatus 2 includes one or more lighting units 31, acamera 32, a filter 33 and a sensor 23. The image capturing apparatus 2has a housing with a substantially cuboidal shape, and an opening 34 forpassing the food item is formed in the bottom part of the housing facingthe carrying apparatus 4. The image capturing apparatus 2 covers apartial area of the carrying apparatus 4 from above. A wall is formed onsome or all of the side surfaces of the housing. Also, a ceiling isformed on the top of the housing. The ceiling and wall can reduce theentrance of infrared light, visible light and the like from externallight sources into the image-capturing region. In this manner, the imagecapturing apparatus 2 can selectively project light from a desired lightsource onto the food item that is the subject of image capture.

Note that the image capturing apparatus 2 may lack one or both of theceiling and the wall. For example, if the image capturing apparatus 2 isprovided in a darkroom, effects from external light sources are notsignificant, and the ceiling and wall for blocking light can be omitted.Examples of the materials of the ceiling and wall include stainlesssteel, aluminum, resin and the like, and the material is notparticularly limited. While in the example of the figure, the imagecapture is performed from directly above the carrying channel, the imagecapture may be performed from obliquely above or in a plurality ofdirections.

The one or more lighting units 31 are provided in a lower part of thehousing of the image capturing apparatus 2. The lighting units 31 areapparatuses that emit ultraviolet light. The irradiation of the chickenfeather with ultraviolet light causes fluorescence to occur, whichenables the detection of the feather. The lighting units 31 irradiatethe image-capturing region below the image capturing apparatus 2 withultraviolet light. In the example of the figure, two lighting units 31are provided. As an example, the two lighting units 31 have a cuboidalshape extending in the direction vertical to the paper plane. However,their shape is not limited to the above. For example, the shape of thelighting units 31 may be cylindrical. Alternatively, there may be asingle lighting unit 31 formed in a ring shape.

Examples of the light source in the lighting unit 31 include anultraviolet LED, a black light, various florescent lamps and the like,and the type of the light source is not particularly limited. In thecase of detecting a chicken feather, an ultraviolet light source havinga spectral peak wavelength within the range from 365 nm to 400 nm ispreferably used as the light source. As an example, an ultraviolet lightsource having a spectral peak wavelength of 375 nm can be used. However,the spectrum of the light source is not particularly limited. A lightsource containing visible light components besides ultravioletcomponents may also be used.

The type of the light source used may be determined according to thetype of the food item to be inspected, the type of the foreign matter ordefect to be detected, and the like. For example, in the case ofdetecting a bone (or cartilage) as foreign matter, X-rays can be used.In general, when a substance is irradiated with electromagnetic waves,the resulting reaction varies with the type of the substance. Forexample, different substances have different wavelengths (frequencies)of electromagnetic waves to absorb, reflect, and emit florescence. Thus,fluorescence spectral analysis may be performed on the food item to beinspected and the foreign matter or defect to be detected, to determinethe wavelengths of electromagnetic waves that evoke strong reaction. Forexample, a fluorescence spectrophotometer can be used to perform thefluorescence spectral analysis.

The camera 32 captures an image of the food item irradiated withultraviolet light emitted from the lighting unit 31. The camera 32 isprovided with, for example, a CMOS image sensor or a CCD image sensormounted therein, and can obtain a color image including colorinformation relating to the food item that is the subject of imagecapture. These image sensors are examples, and the image capture may beperformed using different devices. The camera 32 may have a fixed-focuslens or a zoom lens, and the type of the lens is not particularlylimited. An example of the resolution of an image captured by the camera32 is full HD (1920×1080 pixels), and the resolution of the capturedimage is not particularly limited.

A filter 33 is provided on the front side of the camera 32. The camera32 detects, through the filter 33, fluorescence emitted from the fooditem irradiated with ultraviolet light from the lighting unit 31. Thefilter 33 is provided to allow easy discrimination of foreign matterusing the image captured by the camera 32. In the case of detecting achicken feather as foreign matter, the use of a green filter as thefilter 33 allows easy detection of the feather. That is, a portioncontaining the feather clearly appears and can easily be discriminatedfrom other portions of the chicken. The green filter is a filter thatpasses wavelengths in the wavelength range of green (for example,electromagnetic waves near a wavelength of 520 nm) and blocks otherwavelength ranges. The present embodiment assumes the case of using agreen filter as the filter 33 for performing detection of a feather asforeign matter.

FIG. 3A shows an example of an image of chicken captured without a colorfilter. The region enclosed by an oval in FIG. 3A contains a part of afeather. Referring to FIG. 3A, slightly light-colored lines are includedin the portion containing the feather, but are unclear. FIG. 3B shows anexample of an image obtained through a blue filter. In FIG. 3B as well,the region enclosed by a white oval contains a part of a feather.Although portions around the feather have low luminance, the portioncontaining the feather also has reduced luminance, and the feather isblurred. FIG. 3C shows an example of an image captured through a greenfilter. The region enclosed by a white oval contains a feather. In theimage in FIG. 3C, the portion containing the feather clearly appears andcan easily be discriminated from other portions of the chicken. In lightof these results, it will be understood that performing the imagecapture through the green filter allows easy discrimination of thefeather.

The sensor 23 detects that the food item is carried into the housing ofthe image capturing apparatus 2 by the carrying apparatus 4.Specifically, the sensor 23 detects that the food item enters theimage-capturing region of the image capturing apparatus 2. The sensor 23is arranged at a different position than the food item being carried inthe direction vertical to the paper plane so as not to obstruct thepassage of the food item. Examples of the sensor 23 include a lasersensor, an infrared sensor, an ultrasonic sensor, various photoelectricsensors, an image sensor, a weight sensor and the like, and the type ofthe sensor is not particularly limited.

The display 3 is an information output apparatus that displaysinformation such as an image or a text on the screen 3 a. The display 3is in wired or wireless connection with the inspection assistingapparatus 1. As an interface between the inspection assisting apparatus1 and the display 3, wireless LAN, PCI Express, USB, UART, SPI, SDIO,serial port, Bluetooth, Ethernet and the like can be used. Theseinterfaces are examples, and other manners may also be used. As thedisplay 3, for example, an LCD (liquid crystal display), a CRT(cathode-ray tube), an organic EL (organic electroluminescent) displayand the like can be used, and other types of devices may also be used.The display 3 is arranged at a position and in an orientation such thatthe worker performing the inspection can see the screen 3 a during theinspection. For example, the screen 3 a of the display 3 may bepositioned in front of, obliquely above, obliquely left to, obliquelyright to or obliquely below the worker across the carrying apparatus 4.

The inspection assisting apparatus 1 performs image processing on animage captured by the image capturing apparatus 2, processing ofdetecting foreign matter or a defect, processing of generatinginspection assisting information including information identifying theposition of the detected foreign matter or defect, timing control ofdisplaying the inspection assisting information to the worker, and thelike. As an example, the inspection assisting apparatus 1 is aninformation processing apparatus such as a computing machine providedwith one or more CPUs (central processing units), a storage and acommunication unit and capable of running an OS (operating system) andapplications. Some or all of the functions of the inspection assistingapparatus 1 may be achieved by semiconductor circuits such as an FPGAand an ASIC or a GPU (Graphics Processing Unit). The inspectionassisting apparatus 1 may be a physical computing machine, or may beimplemented by a virtual machine (VM), a container or combinationsthereof. The functions of the inspection assisting apparatus 1 may bedistributed among one or more physical computing machines, virtualmachines and containers. It is also not prohibited to use configurationshaving an increased number of inspection assisting apparatuses 1 forpurposes of improving availability and load distribution.

The inspection assisting apparatus 1 includes an image storage 11, animage processor 12, a controller 13, a detection processor 14 and anassisting information generator 15.

When the controller 13 receives a detection signal from the sensor 23 ofthe image capturing apparatus 2, the controller 13 sends a signal forinstruction of image capture to the camera 32 of the image capturingapparatus 2. That is, the controller 13 instructs the camera 32 toperform image capture at the timing when the food item enters animage-capturing angle range (image-capturing region) of the camera 32.In this manner, automatic image capture of the carried food item isperformed.

The image storage 11 has a storage space to save images captured by thecamera 32 of the image capturing apparatus 2. The image storage 11 maybe a volatile memory such as an SRAM or a DRAM, or may be a non-volatilememory such as an NAND, an MRAM or an FRAM. The image storage 11 mayalso be a storage device such as an optical disc, a hard disk or an SSD.The image storage 11 may be integrated in the inspection assistingapparatus 1, or may be a storage device external to the inspectionassisting apparatus 1. The image storage 11 may also be a removablestorage medium such as an SD memory card or a USB memory.

The image processor 12 performs image processing on a captured image ofthe food item. In the case of detecting a feather, the use of a greenfilter makes the feather clearer than other portions in the capturedimage. In this image processing, processing to further emphasize thefeather is performed. Examples of the image processing includeconversion to the HSV color space, conversion to the RGB space,conversion to grayscale and the like, and there is no limitation to aparticular method. The image processor 12 saves the processed image inthe image storage 11. Note that the saved image may be given anidentifier. The identifier may be a number that increments by a constantvalue, may be a time, or may be determined by other criteria. The imageprocessor 12 may notify the controller 13 of the identifier of theprocessed image or address information of the region in which the imageis stored. It is also possible to perform no image processing on animage captured using the green filter. In that case, the image processor12 may not be provided.

The controller 13 requests the detection processor 14 to performprocessing of detecting foreign matter or a defect on the processedimage saved in the image storage 11. The controller 13 may read theprocessed image from the image storage 11 and provide the read image tothe detection processor 14, or the detection processor 14 may directlyread the processed image from the image storage 11. In this case, thecontroller 13 notifies the detection processor 14 of the identifier ofthe processed image or address information of the region in which theimage is stored. In this example, the processed image is provided to thedetection processor 14 via the controller 13. However, the imageprocessor 12 may provide the processed image to the detection processor14 after performing the image processing.

The detection processor 14 performs processing of detecting foreignmatter or a defect based on the processed image. In this example,processing of detecting a feather is performed. Note that an imageconsists of a plurality of pixels and each pixel has a value within acertain range. For example, if one pixel is represented by 8-bitinformation, the pixel has a value in 256 levels from 0 to 255. Thedetection processor 14 performs edge detection on the processed image toidentify a feather. Specifically, the processed image is first binarizedby a threshold. That is, a binary image is generated by converting pixelvalues greater than or equal to the threshold as 1 and pixel values lessthan the threshold as 0. The relationship between 1 and 0 may bereversed. As a result of the binarization, all or most portions of thefeather become 1, and all or most of other portions become 0. Aconcentrated region of 1 is detected as a feather region.

FIG. 4A shows an example of a binary image obtained by binarizing theprocessed image by a threshold. Black dots correspond to pixels of 1,and other pixels correspond to pixels of 0. There is a short region 30of pixels with values of 1 on the right side of the image. The region 30corresponds to the feather region. In the region 30, all pixels may havevalues of 1, or some pixels may have values of 0. The identification ofthe region 30 can be performed using clustering, for example. Forexample, adjacent pixels with values of 1 within a certain distance onthe image are classified into the same group, and a group including acertain number of pixels or more is identified as a feather region.While there is one feather region in this example, a plurality offeather regions may be identified. The clustering may use anytechniques. Techniques other than clustering may also be used for theregion detection.

The assisting information generator 15 generates informationrepresenting the position of the feather identified by the detectionprocessor 14. Examples of such information include data indicating theposition of the feather. In the present embodiment, frame data enclosingthe feather is generated as the data indicating the position of thefeather. However, other forms of data may be generated, such as data ofan arrow pointing to the feather. The shape of the frame enclosing thefeather may be, for example, a rectangle circumscribing the feather, astar enclosing the feather, or other shapes. The frame desirably has acolor with a higher contrast than the captured image before beingsubjected to image processing (the image captured through the greenfilter). The assisting information generator 15 retains the coordinatesof the generated frame data.

The assisting information generator 15 arranges the generated frame dataat the retained coordinates on the captured image. The resulting image(assisting image) corresponds to an example of inspection assistinginformation. A bilayer image may be formed by placing the frame dataover the captured image, or the values of pixels corresponding to theposition of the frame data may be updated with the values of the framedata. In this manner, an image is obtained in which the feather on thecaptured image is enclosed by the frame data. Note that, if the size ofthe binary image is changed from that of the captured image due to scaleconversion, scale reconversion may be performed to arrange the framedata on the captured image.

The assisting information generator 15 may arrange information otherthan frame data on the captured image. For example, since a feather isdetected as foreign matter, a text or symbol representing a feather maybe arranged near the frame data. For example, text data of “hane(feather)” may be arranged near the frame data. Data representing thesize of the detected feather region may also be arranged near the framedata. The size may be at least one of the horizontal or vertical size ofthe feather region, may be the area (the number of pixels) of thefeather region, or may be a size category such as large, medium orsmall. Data representing the number of identified feather regions mayalso be arranged. For example, if two feather regions are identified,“2” may be arranged on the captured image. In this case, the number ofpieces of frame data is also two. Information other than described abovemay also be arranged.

FIG. 4B shows an example of inspection assisting information generatedby the assisting information generator 15. Frame data with thick whitelines is arranged on the image captured through the green filter, andtext data of “hane (feather)” is arranged near the frame data.

The controller 13 receives the inspection assisting informationgenerated by the assisting information generator 15, and stores it in aninternal buffer or the image storage 11. The controller 13 controlstiming of displaying the inspection assisting information so as todisplay the inspection assisting information at the timing when the fooditem to be inspected passes through the inspection area. Specifically,when the controller 13 receives a detection signal from the sensor 24,the controller 13 reads the generated inspection assisting informationfrom the internal buffer or the image storage 11, and sends the readinspection assisting information to the display 3. The display 3displays the inspection assisting information received from thecontroller 13. When a predetermined period of time (for example, aperiod of time obtained by dividing the length of the inspection area bythe carrying speed) has elapsed after sending the inspection assistinginformation, the controller 13 sends, to the display 3, a stopinstruction signal to stop the displaying of the inspection assistinginformation or a standby image. The standby image is an image for beingdisplayed while the worker waits for the entrance of a next target fooditem into the inspection area. If a plurality of pieces of inspectionassisting information are generated by continuously capturing images ofthe food item, the controller 13 continuously (at constant timeintervals) outputs these pieces of inspection assisting information tothe display 3. The display 3 continuously displays these pieces ofinspection assisting information as a moving image. A sensor may bearranged near the end position of the inspection area such that thesensor sends a detection signal to the controller 13 when detecting thefood item, and the controller 13 sends the stop instruction signal orthe standby image to the display 3 based on the detection signal.

The controller 13 may use any method to determine which piece ofinspection assisting information is to be displayed this time among aplurality of pieces of inspection assisting information sequentiallygenerated for a plurality of food items. For example, the oldest one ofthe pieces of inspection assisting information that is not displayed yetmay be determined as the inspection assisting information to bedisplayed. In this case, flags may be set to discriminate betweeninspection assisting information that is already displayed andinspection assisting information that is not displayed yet.Alternatively, inspection assisting information not displayed may bemanaged by a first-in first-out buffer.

In another method, a sequence number that increments by one for eachchange of the food item for which the image capturing apparatus 2performs image capture is given to inspection assisting informationgenerated for the food item. Also, a sequence number that increments byone each time the sensor 24 detects a food item is also generated.Inspection assisting information having a value equal to the sequencenumber generated for detection by the sensor 24 is determined asinspection assisting information to be displayed this time.

Note that, if food items are carried at long intervals so that a fooditem is in the inspection area and a next food item to be inspected isnot carried to the image-capturing region yet, the inspection assistinginformation generated most lately may be displayed.

The display 3 receives the inspection assisting information from thecontroller 13, and displays the received inspection assistinginformation (refer to FIG. 4B) on the screen 3 a. When receiving a stopinstruction signal from the controller 13, the display 3 stopsdisplaying the inspection assisting information. When receiving astandby image, the display 3 displays the standby image.

FIG. 5A and FIG. 5B show a specific configuration example of the system.FIG. 5A is a side view of the system. FIG. 5B is a plan view of thesystem. In FIG. 1 as described above, the left side is the upstream sideand the right side is the downstream side as seen toward the paperplane. In FIG. 5A and FIG. 5B, the right side is the upstream side andthe left side is the upstream side as seen toward the paper plane. InFIG. 5A, the image capturing apparatus 2, the carrying apparatus 4, thecollecting apparatus 4 a, the display 3 and a food item 21 are shown. InFIG. 5B, the image capturing apparatus 2, the carrying apparatus 4, thecollecting apparatus 4 a, the display 3, the food item 21, the sensor23, the sensor 24, the inspection assisting apparatus 1 and a worker 27are shown. The following describes a flow of food inspection using thesystem with reference to FIG. 5A and FIG. 5B.

The food item (chicken) 21 is placed on the conveyor belt of thecarrying apparatus 4 on the upstream side, and is carried toward theimage capturing apparatus 2. When the food item 21 enters theimage-capturing region below the housing of the image capturingapparatus 2, the food item 21 is detected by the sensor 23, and imagecapture of the food item 21 is performed by a camera in the housing ofthe image capturing apparatus 2. After the image capture, the food item21 exits the housing of the image capturing apparatus 2, and is carriedtoward the downstream side. Meanwhile, the captured image is sent to theinspection assisting apparatus 1, the inspection assisting apparatus 1detects a feather based on the image, and generates inspection assistinginformation including information identifying the position of thedetected feather (in this example, an assisting image in whichrectangular frame data 5 representing the position of the feather isplaced over the image of the chicken).

When the food item 21 after exiting the housing of the image capturingapparatus 2 is detected by the sensor 24, the inspection assistingapparatus 1 outputs the generated inspection assisting informationdescribed above to the display 3. The display 3 is supported by an armmember 28 such that the screen is vertically oriented to the floor alongone side surface of the carrying apparatus 4. The display 3 displays theinspection assisting information input from the inspection assistingapparatus 1 on the screen 3 a. Meanwhile, the worker 27 inspects thefood item 21 carried to the inspection area by visual check or palpationor by both. Note that, during the inspection, the food item 21 may beheld on the carrying apparatus 4, or the food item 21 may be lifted byhand from the carrying apparatus 4. The worker 27 searches the food item21 for a feather while referring to the inspection assisting informationdisplayed on the display 3 to check the position and size of the framedata 5. The worker removes the feather found. After the feather isremoved, the food item is carried by the carrying apparatus 4 toward thedownstream side from the inspection area, and is collected by thecollecting apparatus 4 a.

FIG. 6 is a flowchart of an example of operations according to anembodiment of the present invention. When the sensor 23 detects that afood item to be inspected enters the image-capturing region of the imagecapturing apparatus 2 (step S101), the sensor 23 sends a detectionsignal to the controller 13. The controller 13 outputs a signal forinstruction of image capture to the camera 32. The camera 32 captures animage of the food item through a green filter, and the controller 13stores the captured image in the image storage 11 (S102). The imageprocessor 12 performs image processing on the captured image (S103).

The detection processor 14 performs processing of detecting foreignmatter by using edge detection based on the processed image (S104). Inthis example, processing of detecting a feather as foreign matter isperformed. Specifically, the processed image is first binarized by athreshold to generate a binary image. Based on the binary image, aregion of pixels of 1 (or 0) is identified as a region of feather.

The assisting information generator 15 generates informationrepresenting the identified region of feather (S105). For example, theassisting information generator 15 generates frame data enclosing thefeather, and generates an image in which the frame data is arranged onthe captured image (inspection assisting information).

The controller 13 receives the generated inspection assistinginformation, and displays the inspection assisting information on thedisplay 3 at the timing when the food item is carried in the inspectionarea (S106). If there are a plurality of pieces of inspection assistinginformation to be displayed, these pieces of inspection assistinginformation are continuously output to sequentially switch thedisplaying of these pieces of inspection assisting information. In thismanner, the food item and the inspection assisting information displayedon the display 3 appear to move as a moving image in accordance with themovement of the food item on the carrying channel.

While in the present embodiment, the region of foreign matter isidentified by performing edge detection on the binary image, othermethods may also be used. For example, a model may output information ofthe position or region of foreign matter by using captured images orprocessed images as inputs to the model. In this case, the model isconstructed by machine learning by using data in which images (capturedimage or processed images) to be a sample and positions or regions offoreign matter are associated with each other. An example of the modelis a neural network.

While in the present embodiment, the sensor 24 detects that the fooditem is carried to the inspection area, that can be detected by othermethods. For example, the timing when the food item appears in theinspection area may be calculated based on the carrying speed of thefood item and the distance from the position of capturing the image ofthe food item (or the position of detection by the sensor 23) to theinspection area. Any method may be used as long as it detects that thefood item is carried to the inspection area based on the time requiredto carry the food item to the inspection area.

According to the present embodiment, accurate and quick inspection offood raw materials can be achieved, and production of high-quality foodsand cost reduction can be realized.

(Modified Example 1)

In the first embodiment, a region of feather is identified using abinary image. However, if the food item is a quality item, no feather iscontained and a region of feather will not be found. In that case, theassisting information generator 15 generates information indicating thatno feather is contained (the food item is a quality item) as inspectionassisting information. For example, text data of “NO Hane (Feather)” or“GOOD” is generated. The worker can determine that the food item to beinspected contains no feather by checking the information on the display3, and the operation of searching for a feather can be omitted.

(Modified Example 2)

As an extension of the modified example 1, the inspection assistingapparatus 1 may determine the food item as a defective item if a featheris detected and determine the food item as a quality item if no featheris detected, and generate information representing whether the food itemis a defective item or quality item as inspection assisting information.The worker collects only food items for which information of qualityitem is displayed in the collecting apparatus 4 a, and picks up fooditems determined as defective items from the carrying apparatus 4 andcollects them in a separate container. A sorting apparatus thatautomatically sorts food items according to whether it is a quality itemor defective item may be provided to the carrying apparatus 4 such thatthe sorting apparatus automatically performs the sorting. In this case,the sorting apparatus receives information of a determination result ofwhether the food item is a quality item or defective item from theinspection assisting apparatus 1, and performs sorting based on thereceived information of the determination result.

Second Embodiment

In the second embodiment, blood-containing part is detected from a fooditem as a defect. The blood-containing part not only refers to a meatportion that is colored due to a bruise or a blood clot, but alsoincludes a colored portion having a different color, taste or the likethan other portions. Other examples of defects include those relating toshape such as chipping, cracking, kinks and the like. The systemconfiguration is the same as that in the first embodiment in principle.The following describes differences from the first embodiment.

FIG. 7 shows an example of a captured image of chicken according to thepresent embodiment. The portion enclosed by a circle containsblood-containing part. In the present embodiment, such blood-containingpart is detected as a defect of the food item (chicken).

FIG. 8 is a cross-sectional view of an image capturing apparatus 51according to the present embodiment. In the present embodiment, theimage capturing apparatus 51 is used instead of the image capturingapparatus 2 shown in FIG. 1 .

The image capturing apparatus 51 includes one or more lighting units 52,a polarizing plate 53, a camera 55, a polarizing plate 56 and a sensor23. The image capturing apparatus 51 has a housing with a substantiallycuboidal shape, and an opening 54 for passing the food item is formed inthe bottom part of the housing facing the carrying apparatus 4. Thespecific structure of the housing is similar to that in the firstembodiment, and the description thereof is omitted.

The lighting units 52 are lighting apparatuses using white LEDs, forexample. A plurality of lighting units 52 are provided, and the lightingunits 52 on the left and right have a shape extending in the directionvertical to the paper plane. The lighting unit 52 at the center includesone or more lighting units with a shape extending in the left and rightdirections along the paper plane, and the one or more lighting units arearranged at intervals in the direction vertical to the paper plane.Light passing through the intervals is captured by the camera 55 throughthe polarizing plate 56.

The white LED is an example, and LEDs of different colors may also beused as the light source. For example, a light source may be constructedby combining LEDs of a plurality of colors such as red, green and blue.Other light sources such as an electroluminescence lamp, an HID lamp, afluorescent lamp and an incandescent lamp may also be used. The type ofthe light source used can be determined according to the type of thefood item or foreign matter to be detected. The light source is notlimited to a light source that emits visible light, and may be a lightsource that emits infrared light, ultraviolet light or the like, or maybe a combined light source of a plurality of wavelengths(multi-wavelength).

The polarizing plate 53 is provided on the front side of the lightingunit 52. For example, the polarizing plate 53 is a linear polarizingplate. Providing the polarizing plate 53 reduces diffuse reflection oflight from the surface of the carrying apparatus 4 or water or oils andfats in the food item, and contributes to accurate image capture of thefood item by the camera 55. Configurations without the polarizing plate53 are also possible.

The camera 55 captures an image of the food item irradiated with beamsof light from the lighting unit 52. The polarizing plate 56 is providedon the front side of the camera 55. For example, the polarizing plate 56is a polarizing filter for camera lenses. By performing image capturethrough the polarizing plate 53, an image can be obtained with reducedeffects of diffuse reflection or sheens due to the luster of water, oilsand fats and the like. Configurations without the polarizing plate 56are also possible.

The image processor 12 extracts the luminance of each color of red,green and blue (RGB) for the pixels in a captured image, which is outputinformation of the image capturing apparatus 51, and obtains an imagefor each color. The image processor 12 then generates an image bysubtracting the pixel values of the green (G) image from the red (R)image for each pixel. The image thus generated is subjected tobinarization processing using a threshold to generate a binary image, ina manner similar to that in the first embodiment.

FIG. 9 shows a specific example of image processing. A captured image 61is separated into an R image 62, a G image 63 and a B image 64. Thepixel values of the G image 63 are subtracted from the pixel values ofthe R image 62 to generate an image (R-G image). The generated image issubjected to binarization processing to generate a binary image 65. Thecaptured image 61 is a color image, and the images 62 to 64 aregrayscale images expressing the magnitudes of luminance for respectivecolors of R, G and B. In the binary image 65, pixels with values greaterthan or equal to the threshold are white, and pixels with values lessthan the threshold are black. A region of pixels with values greaterthan or equal to the threshold corresponds to a portion ofblood-containing part.

The detection processor 14 can identify the region of blood-containingpart (a white region in the binary image 65) by performing edgedetection on the binary image 65 in a manner similar to that in thefirst embodiment.

Note that the image processing described above is an example, and othertypes of image processing may be performed. For example, afterperforming processing to emphasize the red and green colors on acaptured image, the captured image on which the processing has beenperformed may be separated into an R image, a G image and a B image.Besides, after performing conversion (for example, conversion to the HSVcolor space) of an image obtained by subtraction (R-G image), an imagemay be generated by extracting the red components from the convertedimage. An image may also be generated by extracting the red componentsfrom the R-G image without converting the R-G image. Note that it isalso not prohibited to perform binarization processing on a capturedimage without performing any image processing on the captured image.

The assisting information generator 15 generates inspection assistinginformation including information identifying the position ofblood-containing part, in a manner similar to that in the firstembodiment. For example, the assisting information generator 15generates frame data enclosing the region of blood-containing part basedon the binary image, and arranges the generated frame data on thecaptured image. Text data such as “blood-containing part” may bearranged near the frame data. The image thus generated (assisting image)is displayed on the display 3 as inspection assisting information. Thecontrol of display timing is similar to that in the first embodiment.The worker inspects the food item carried into the inspection area whilereferring to the frame data as inspection assisting informationdisplayed on the display 3 to check the position of the blood-containingpart. When finding the blood-containing part, the worker removes theblood-containing part using a predetermined tool. Alternatively, theworker may check the blood-containing part so that, if theblood-containing part is of an allowable extent (e.g., small), theworker allows the food item to be collected by the collecting apparatus4 a, and if the blood-containing part is not of an allowable extent, theworker stores it in a predetermined case for separately performing theoperation of removing the blood-containing part in another step. Othermethods may be used to deal with chicken for which blood-containing partis detected.

(Modified Example)

In the second embodiment described above, an R-G image is generated anda binary image is generated from the R-G image as image processing(refer to FIG. 9 ). In this modified example, an infrared (INFRARED)image is captured in addition to a color image, and the pixel values ofthe G image are subtracted from the pixel values of the infrared imageto generate an image. A specific example of the infrared image is anear-infrared (NEAR INFRARED) image. Near-infrared light iselectromagnetic waves with wavelengths between 760 nm and 1000 nm, forexample. In this modified example, a near-infrared image is captured,and the pixel values of the G image are subtracted from the pixel valuesof the near-infrared image to generate an image (NIR-G image). Further,the NIR-G image is subjected to binarization processing to generate abinary image. This binary image is used to detect foreign matter or adefect. In this manner, the accuracy of detecting darkerblood-containing part (for example, highly blackish blood-containingpart) can be improved.

In this modified example, a camera capable of capturing both colorimages and near-infrared images is used as the camera 55. As an exampleof such a camera, the combination of both an RGB camera and an infraredcamera may be used. Alternatively, a multispectral camera or ahyperspectral camera capable of capturing images of a larger number ofwavelengths may be used. The configuration, number and type of thecamera are not particularly limited. If both of an RGB camera and aninfrared camera are used, image capture may be performed at the sametime or in an order. In the latter case, the image processor 12 at alater step may perform alignment of the images for performing thesubtraction processing on the images.

The color image and near-infrared image of the food item (chicken)captured by the camera 55 are sent to the image storage 11. The imagestorage 11 stores the color image and near-infrared image therein. Theimage processor 12 reads the color image from the image storage 11, andextracts the luminance of each color of red, green and blue (RGB) fromthe color image, to generate an R image (red image) 131, a G image(green image) 132 and a B image (blue image) 133.

FIG. 10 shows an example of the R image 131, G image 132 and B image 133generated by the image processor 12. The figure also shows an example ofa near-infrared image 134 stored in the image storage 11. In the images131 to 134, the intensity of electromagnetic waves of respectivewavelengths is represented by a grayscale tone (luminance) for eachpixel. A pixel with a higher intensity is shown to be brighter. A pixelwith a smaller intensity is shown to be darker. In any of the images 131to 134, the portion of meat stands out with brighter colors than thesurrounding portions. This example uses a sample containing only highlyblackish blood-containing part.

The image processor 12 generates an image (NIR-G image) by subtractingthe pixel values of the G image 132 from the near-infrared image 134 foreach pixel, that is, generates a difference image between thenear-infrared image 134 and the G image 132. The image processor 12performs binarization processing on the NIR-G image to generate a binaryimage (NIR-G binary image) 135. The detail of the binarizationprocessing is described above.

The upper side of FIG. 11 shows an example of the NIR-G binary image 135generated by the image processor 12. The lower side of FIG. 11 shows anexample of a binary image (R-G binary image) 136 obtained by performingbinarization processing on an image (R-G image) obtained by subtractingthe pixel values of the G image 132 from the R image 131 for each pixel.The R-G image corresponds to the difference image between the R image131 and the G image 132. The R-G binary image 136 is shown forcomparison with the NIR-G binary image 135, and it is not necessarilyrequired to generate the R-G binary image 136 in this modified example.

In the NIR-G binary image 135 and R-G binary image 136, pixels withvalues greater than or equal to the threshold are white, and pixels withvalues less than the threshold are black. A region of pixels with valuesgreater than or equal to the threshold corresponds to theblood-containing part.

Referring to the R-G binary image 136 on the lower side of the FIG. 11 ,while the pixels of the portion of highly blackish blood-containing partare detected in a region 139 enclosed by a circle, a lot of noisecomponents appear at portions where actually no blood-containing part iscontained, as shown in a region 138 enclosed by a circle. This isbecause a low threshold is used for generating the R-G binary image 136in order to detect highly blackish blood-containing part.

On the other hand, in the NIR-G binary image 135 on the upper side ofFIG. 11 , while the pixels of the portion of highly blackishblood-containing part are detected in a region 137 enclosed by a circle,only a little amount of noise components appear in the regioncorresponding to the region 138 of the R-G binary image 136. This isbecause the NIR-G image can emphasize the portion of blackishblood-containing part, and it is not necessary to lower the thresholdfor generating the NIR-G binary image 135 as much as for the R-G binaryimage 136. That is, a higher threshold is used for the NIR-G binaryimage 135 than for the R-G binary image 136. Thus, the use of the NIR-Gbinary image 135 enables both of reduction of noise components andenhancement of the accuracy of detecting highly blackishblood-containing part.

In this modified example, other types of images may be used instead of anear-infrared image. For example, images of other types of infraredlight than near-infrared light (such as far-infrared light), ultravioletimages or three-dimensional images may also be used. Images of the othertypes of infrared light can be captured by using, for example, a cameraprovided with a sensor with sensitivity to the other types of infraredlight. Ultraviolet images can be captured by using, for example, acamera provided with a sensor with sensitivity to the ultraviolet light.Three-dimensional images can be captured by using, for example, a 3Dcamera.

Also, while operation (subtraction) is performed between thenear-infrared image and the G image in this modified example, thecombination of images to be operated is not limited to the combinationof the near-infrared image and the G image. That is, combinations of anytypes of images can be subjected to the operation as long as the foreignmatter or defect to be detected can be accurately detected. For example,two or more of a near-infrared image, an image of other types ofinfrared light, an ultraviolet image, a three-dimensional image, an Rimage, a G image and a B image can be subjected to the operation. Thetype of the operation is not limited to subtraction, and may also beaddition, subtraction, multiplication, division or combinations thereof.The images may also be multiplied by weighting coefficients beforeperforming the operation.

The modified examples of the first embodiment are similarly applicableto the present embodiment.

Third Embodiment

While the first and second embodiments use the display 3 as aninformation output apparatus that outputs inspection assistinginformation, the third embodiment uses a projector, which is an exampleof a projection apparatus, to project the inspection assistinginformation onto the food item being carried or onto the carryingchannel. The following describes the present embodiment in details.

FIG. 12 shows an example of a food inspection assisting system accordingto the third embodiment. A projector 71 is used in place of the display3 in FIG. 1 . The projector 71 is provided above the carrying channel inthe inspection area, and projects inspection assisting informationtoward the carrying surface. Other configurations are similar to thosein the first embodiment in principle. The following mainly describeschanges or extensions from the first embodiment.

FIG. 13A illustrates an example of continuous image capture performed bythe image capturing apparatus 2. When chicken (food item) 21 carried bythe carrying apparatus 4 enters an image-capturing region 81 in theimage capturing apparatus 2, a plurality of times of image capture(continuous shooting of static images) is performed at constant timeintervals. The figure shows an example in which image capture isperformed three times in total at time t11, time t12 and time t13. Sincethe food item moves even during the plurality of times of image capture,the position of the food item in the captured images gradually changesin the carrying direction. While three times of image capture isperformed in this example, the number of times or rate of performingimage capture may be determined as required.

FIG. 14 shows an example of captured images obtained by continuous imagecapture. FIG. 14A shows an image 91 captured for a first time at timet11, FIG. 14B shows an image 92 captured for a second time at time t12,and FIG. 14C shows an image 93 captured for a third time at time t13. Inthe captured images 91 to 93, the food item that is the subject of imagecapture gradually moves in the carrying direction.

The image capturing apparatus 2 sends, to the inspection assistingapparatus 1, the plurality of captured images obtained by the continuousimage capture. The inspection assisting apparatus 1 generates aplurality of pieces of inspection assisting information for theplurality of captured images in a manner similar to that in the firstembodiment. In this example, each piece of inspection assistinginformation is obtained by arranging frame data on each captured image,in a manner similar to that in the first embodiment.

When the controller 13 receives a detection signal for the food itemfrom the sensor 24, the controller 13 controls the projector 71 suchthat the plurality of pieces of inspection assisting information arecontinuously (for example, at constant time intervals) projected. Thecontroller 13 outputs the plurality of pieces of inspection assistinginformation to the projector 71 at constant time intervals. Theprojector 71 sequentially projects the plurality of pieces of inspectionassisting information input from the controller 13 onto the carryingchannel at constant time intervals. For example, the size of theirradiation region of the projector 71 is the same as that of theimage-capturing region of the image capturing apparatus 2.

FIG. 13B shows an example of projection performed by the projector 71.After exiting the housing of the image capturing apparatus 2, thechicken (food item) 21 is carried by the carrying apparatus 4 and entersa projection region 82 of the projector 71. At this time, the pluralityof pieces of inspection assisting information are projected by theprojector 71 at constant time intervals. FIG. 15 shows an example of aplurality of pieces of inspection assisting information continuouslyprojected. FIG. 15A shows inspection assisting information 101 projectedfor a first time at time t21, FIG. 15B shows inspection assistinginformation 102 projected for a second time at time t22, and FIG. 15Cshows inspection assisting information 103 projected for a third time attime t23. The respective pieces of information correspond to thecaptured images of FIG. 14A to FIG. 14C. In the inspection assistinginformation 101 to 103, the food item gradually moves in the carryingdirection, and frame data 75 moves correspondingly in the carryingdirection by the same amounts. The pieces of inspection assistinginformation of FIG. 15A to FIG. 15B are sequentially projected, so thatthe inspection assisting information (food item images and frame data)is projected over the food item being carried, as shown in FIG. 13B. Thefood item moves even during the image capture by the projector 71, andthe food item and frame included in the inspection assisting informationprojected at time t21, time t22 and time t23 also move at the samespeed. Thus, the images of the food item and frame included in theinspection assisting information are projected as conforming to theposition of the food item regardless of the movement of the food item.Therefore, the frame projected on the carried food item contains foreignmatter (feather). The worker can see that the feather is contained inthe frame projected on the food item, and can easily find the feather.

FIG. 16 is a flowchart of an example of operations according to anembodiment of the present invention. When the sensor 23 detects that afood item to be inspected enters the image-capturing region of the imagecapturing apparatus 2 (step S201), the sensor 23 sends a detectionsignal to the controller 13. The controller 13 outputs a signal forinstruction of image capture to the camera 32. The camera 32 performs aplurality of times of image capture (continuous image capture) of thefood item at constant time intervals through a green filter, and thecontroller 13 stores the plurality of captured images in the imagestorage 11 (S202). The image processor 12 performs image processing oneach captured image. Specifically, the image processor 12 generates animage by performing image processing on each captured image (S203).

The detection processor 14 performs processing of detecting foreignmatter by using edge detection based on the processed image (S204). Inthis example, processing of detecting a feather as foreign matter isperformed. Specifically, the detection processor 14 first binarizes theprocessed image by a threshold to generate a binary image. Based on thebinary image, a region of pixels of 1 (or 0) is identified as a featherregion.

The assisting information generator 15 generates informationrepresenting the position of the identified feather region (S205). Forexample, the assisting information generator 15 generates frame dataenclosing the feather, and generates an image in which the frame data isarranged on the captured image (inspection assisting information).

The controller 13 receives the plurality of pieces of inspectionassisting information generated for the plurality of captured images,and sequentially outputs the plurality of pieces of inspection assistinginformation to the projector 71 at constant time intervals at the timingwhen the food item to be inspected passes through the inspection area.In this manner, the plurality of pieces of inspection assistinginformation are projected onto the food item being carried on thecarrying channel at constant time intervals (S206). This allows theposition of the frame data included in the projected inspectionassisting information to be moved in accordance with the movement of thefood item on the carrying channel, so that the worker can continuechecking the frame projected on the foreign matter (feather) in the fooditem without interruption. Thus, the worker can easily identify andremove the feather.

While in the present embodiment, the image-capturing region(image-capturing direction) of the camera 32 and the projection region(projection direction) of the projector 71 are fixed, in the case ofperforming continuous image capture of the food item, the direction ofthe camera 32 may be moved while capturing images of the food item.Similarly, the projection direction of the projector 71 may be movedwhile sequentially projecting the plurality of pieces of inspectionassisting information. This allows the use of a camera with a narrowangle of view and a projector with a narrow angle of projection.

In the present embodiment, images in which frame data is arranged oncaptured images are projected onto the food item as inspection assistinginformation. However, images including only frame data (not includingchicken) may be generated as inspection assisting information andprojected onto the food item.

While in the present embodiment, the inspection assisting information isprojected onto the food item, the inspection assisting information mayalso be projected onto the carrying channel. For example, the inspectionassisting information may be projected onto a part of the carryingchannel near (for example, on the right of, on the left of, above orbelow) the food item being carried. In this case as well, the positionof the projected information changes in synchronization with the fooditem being carried. In this manner, the worker can perform inspectionwhile looking at the inspection assisting information displayed near thefood item being carried.

While in the present embodiment, a projector that projects informationby light is used as an information output apparatus, a beam projectorthat projects information by light beams may also be used. In the caseof using a beam projector, light beams may be projected onto theposition of detected foreign matter. The projected beams may also bemoved to enclose the foreign matter. In any case, the position of theprojected beams changes in synchronization with the food item beingcarried. Such projection of light beams is also a form of displayinginspection assisting information.

While the present embodiment has been described by using an example ofdetecting foreign matter (feather) based on the first embodiment, thesame may apply to embodiments of detecting a defect (such asblood-containing part) such as the second embodiment.

Fourth Embodiment

In the first to third embodiments, processing of detecting foreignmatter or a defect in a food item is performed based on captured images,and inspection assisting information obtained according to the result ofdetection is generated and displayed. In the present embodiment,characteristic information of a food item is measured, and inspectionassisting information obtained according to the measured characteristicinformation is generated and displayed. Other points are similar tothose in the above-described embodiments in principle.

FIG. 17 shows an example of a food inspection assisting system accordingto the present embodiment. The image capturing apparatus 2 in FIG. 12used in the third embodiment is replaced with a three-dimensional camera(3D camera) 112, and the detection processor 14 is replaced with ameasurement processor 111. Elements with the same names as in FIG. 12are provided with the same reference numerals. The following mainlydescribes changes and extensions from the third embodiment.

The 3D camera 112 captures an image of a food item to be inspected. The3D camera 112 is an example of a sensing apparatus. The 3D camera 112measures the color and depth of the food item, and acquires a 3D image,which is output information of the 3D camera 112, as a captured image.The 3D image includes the color information and depth for each pixel.The depth can be measured using an LED that emits infrared light and asensor that receives reflected infrared light, for example. When thesensor 23 detects the food item 122, the sensor 23 outputs a detectionsignal to the controller 13. The controller 13 receives the detectionsignal and outputs a signal for instruction of image capture to the 3Dcamera 112, and the 3D camera 112 performs image capture. The number oftimes of image capture is one or more. The following description assumesthe case of performing a plurality of times of image capture (continuousimage capture).

The image processor 12 performs preprocessing on the 3D image. Forexample, the image processor 12 performs processing of reducing noisecomponents (averaging), contrast adjustment, tone adjustment and thelike. Configurations without performing any preprocessing are alsopossible.

The measurement processor 111 measures characteristic information of thefood item based on the 3D image. For example, the measurement processor111 measures the weight of the food item. The weight is calculated bymeasuring the volume of the food item based on the 3D image andmultiplying the volume by an averaged density of the food item(chicken). The averaged density of the chicken is provided to themeasurement processor 111 in advance, or is stored in a storage unitsuch as a memory accessible to the measurement processor 111. Besidesthe weight, the size (at least one of the length, width and height) ofthe chicken may also be measured. The fat proportion of the chicken mayalso be calculated. For example, all portions of pixels with luminancevalues greater than or equal to a predetermined value (whitish parts ofthe chicken) in the depth direction are regarded as fat portions, andthe fat proportion is calculated from the proportion of the entirevolume of the chicken and the volume of the fat portions. Note that,while the present embodiment uses a 3D camera to measure thecharacteristic information of the food item, visible light orultraviolet light can also be used to perform the measurement. Also,instead of the 3D camera, a weight sensor, which is a sensing apparatus,may be arranged in the carrying apparatus 4 so that the weight sensordetects the weight of the chicken.

The assisting information generator 15 generates inspection assistinginformation based on the characteristic information of the food itemmeasured by the measurement processor 111. Specifically, the assistinginformation generator 15 generates an image in which the characteristicinformation of the food item is arranged at the position of (placedover) the chicken in a three-dimensional image or a two-dimensionalimage obtained by removing depth information. Alternatively, an image inwhich the characteristic information is arranged at the same position asthe chicken and no image of chicken is contained may be generated.

The controller 13 receives a plurality of pieces of inspection assistinginformation generated for a plurality of captured images (3D images).When the controller 13 receives a detection signal for the food item 121from the sensor 24, the controller 13 controls the projector 71 suchthat the plurality of pieces of inspection assisting information areprojected at constant time intervals. The controller 13 outputs theplurality of pieces of inspection assisting information to the projector71 at constant time intervals. The projector 71 sequentially projectsthe plurality of pieces of inspection assisting information input fromthe controller 13 onto the carrying channel at constant time intervals.For example, the size of the irradiation region of the projector 71 isthe same as that of the image-capturing region of the image capturingapparatus 2.

FIG. 18 shows an example of projection performed by the projector 71.The chicken (food item) 121 is carried by the carrying apparatus 4 andenters a projection region 82 of the projector 71. At this time, theplurality of pieces of inspection assisting information are projected bythe projector 71 at constant time intervals. Similar to the thirdembodiment, the inspection assisting information (including thecharacteristic information of the food item) is projected over the fooditem being carried. In the example of the figure, text data of “398grams” is projected onto the chicken 121 to be inspected. The food item121 moves even during the image capture by the projector 71, and thecharacteristic information included in the inspection assistinginformation projected at time t31, time t32 and time t33 movesaccordingly. The characteristic information is projected over the fooditem regardless of the movement of the food item. The worker determinesif the weight of the chicken is in a range defined by the specificationsby checking the value of the weight projected on the food item. If theweight is in the range defined by the specifications, the chicken isregarded to be qualified and is allowed to be collected by thecollecting apparatus 4 a. If the weight is out of the specificationrange, the chicken is picked up from the carrying channel by hand, andis contained in a case separately prepared. In the example of thefigure, the weight is in the specification range, and the workerdetermines that there is no problem and lets the chicken 121 go. Thepieces of chicken collected in the case are subjected to weightadjustment in a later operation to fall within the specification range.Alternatively, and conversely, pieces of chicken that fall within thespecification range may be picked up and contained in a case separatelyprepared, and pieces of chicken out of the specification range may becollected by the collecting apparatus 4 a.

FIG. 19 is a flowchart of an example of operations according to theembodiment of the present invention. When the sensor 23 detects that afood item to be inspected enters the image-capturing region of the imagecapturing apparatus 2 (step S301), the sensor 23 sends a detectionsignal to the controller 13. The controller 13 outputs a signal forinstruction of image capture to the 3D camera 112. The 3D camera 112performs a plurality of times of image capture (continuous imagecapture) at constant time intervals, and the controller 13 stores theplurality of captured images (3D images) in the image storage 11 (S302).The image processor 12 performs image processing (processing of reducingnoise components) on each captured image (S303).

The detection processor 14 measures characteristic information of thefood item based on the processed image (S304). For example, thedetection processor 14 measures at least one of the weight, size and fatproportion of the food item.

The assisting information generator 15 generates inspection assistinginformation including the measured characteristic information (S305).

The controller 13 receives the plurality of pieces of inspectionassisting information generated for the plurality of captured images,and sequentially outputs the plurality of pieces of inspection assistinginformation to the projector 71 at constant time intervals at the timingwhen the food item to be inspected is carried in the inspection area(S306). The projector 71 projects the plurality of pieces of inspectionassisting information at constant time intervals (also S306). Thisallows the position of the characteristic information included in theprojected inspection assisting information to be moved in accordancewith the movement of the food item on the carrying channel, so that thecharacteristic information is projected on the food item in accordancewith the movement of the food item being carried. The worker cancontinue checking the information (such as weight information) projectedon the food item without interruption. The various extended examples andmodified examples described in the third embodiment are also applicableto the present embodiment.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

REFERENCE SIGNS LIST

1 food inspection assisting apparatus

2 image capturing apparatus

3 display

4 conveying apparatus

4 a collecting apparatus

5 frame data

11 image storage

12 image processor

13 controller

14 detection processor

15 assisting information generator

21, 122, 121 chicken (food)

21 a foreign matter (feather)

23, 24 sensor

27 worker

28 arm member

30 region

31, 52 lighting unit

32, 55 camera

33 filter

34, 54 opening

53, 56 polarizing plate

71 projector

82 projection region

111 measurement processor

112 3D camera

The invention claimed is:
 1. A food inspection assisting apparatuscomprising: a first circuitry configured to acquire a captured image ofa food item being carried by a conveyor from a first sensor irradiatingthe food item with ultraviolet light and detecting generatedfluorescence through a green filter, generate a binary image based onthe image captured through the green filter, and detect a foreign matteror defect included in the food item based on the binary image; a secondcircuitry configured to generate an inspection assisting information forthe food item based on the captured image, wherein the inspectionassisting information includes an information identifying a position ofthe detected foreign matter or a position of the detected defect; and athird circuitry configured to control display timing of the inspectionassisting information to be displayed by an information output devicebased on time required to convey the food item to an inspection area inwhich the food item is inspected.
 2. The food inspection assistingapparatus according to claim 1, wherein the second circuitry is furtherconfigured to generate an indicating data of a position of the detectedforeign matter or the detected defect, and generate the inspectionassisting information including an image in which the indicating data isarranged at the position of the detected foreign matter or the detecteddefect.
 3. A food inspection assisting system comprising: the foodinspection assisting apparatus according to claim 1; the conveyerconfigured to carry the food item on a carrying surface to theinspection area; and the information output device which includes afourth circuitry configured to display the inspection assistinginformation; wherein the third circuitry is further configured to move aposition of displaying the inspection assisting information insynchronization with movement of the food item carried in the inspectionarea.
 4. A food inspection assisting system comprising: the foodinspection assisting apparatus according to claim 1; the conveyerconfigured to carry the food item on a carrying surface to theinspection area; and the information output device which comprises adisplay that displays the inspection assisting information on a screen,or a projector that projects the inspection assisting information to acarrying channel in the inspection area or to the food item carried inthe inspection area.
 5. A food inspection assisting system comprising:the food inspection assisting apparatus according to claim 1; theconveyer configured to carry the food item on a carrying surface to theinspection area; the information output device configured to display theinspection assisting information; and a second sensor that detects apassage of the food item, wherein the third circuitry is furtherconfigured to perform control to display the inspection assistinginformation when the food item is detected by the second sensor.
 6. Afood inspection assisting apparatus comprising: a first circuitryconfigured to acquire a color image of a food item from a first sensorcapturing an image of the food item being carried by a conveyor,generates a difference image between a red image and a green imageincluded in the color image, and detects a foreign matter or defectincluded in the food item based on the difference image; a secondcircuitry configured to generate an inspection assisting information forthe food item based on the captured image, the inspection assistinginformation includes an information identifying a position of thedetected foreign matter or a position of the detected defect; and athird circuitry configured to control display timing of the inspectionassisting information to be displayed by an information output devicebased on time required to convey the food item to an inspection area inwhich the food item is inspected.
 7. The food inspection assistingapparatus according to claim 6, wherein the second circuitry is furtherconfigured to generate an indicating data of a position of the detectedforeign matter or the detected defect, and generate the inspectionassisting information including an image in which the indicating data isarranged at the position of the detected foreign matter or the detecteddefect.
 8. A food inspection assisting system comprising: the foodinspection assisting apparatus according to claim 6; the conveyerconfigured to carry the food item on a carrying surface to theinspection area; and the information output device which includes afourth circuitry configured to display the inspection assistinginformation; wherein the third circuitry is further configured to move aposition of displaying the inspection assisting information insynchronization with movement of the food item carried in the inspectionarea.
 9. A food inspection assisting system comprising: the foodinspection assisting apparatus according to claim 6; the conveyerconfigured to carry the food item on a carrying surface to theinspection area; and the information output device which comprises adisplay that displays the inspection assisting information on a screen,or a projector that projects the inspection assisting information to acarrying channel in the inspection area or to the food item carried inthe inspection area.
 10. A food inspection assisting system comprising:the food inspection assisting apparatus according to claim 6; theconveyer configured to carry the food item on a carrying surface to theinspection area; the information output device configured to display theinspection assisting information; and a second sensor that detects apassage of the food item, wherein the third circuitry is furtherconfigured to perform control to display the inspection assistinginformation when the food item is detected by the second sensor.
 11. Afood inspection assisting apparatus comprising: a first circuitryconfigured to acquire an infrared image and a color image of a food itemfrom a first sensor capturing an image of the food item being carried bya conveyor, generates a difference image between the infrared image anda green image included in the color image, and detects a foreign matteror defect included in the food item based on the difference image; asecond circuitry configured to generate an inspection assistinginformation for the food item based on the captured image, wherein theinspection assisting information includes an information identifying aposition of the detected foreign matter or a position of the detecteddefect; and a third circuitry configured to control display timing ofthe inspection assisting information to be displayed by an informationoutput device based on time required to convey the food item to aninspection area in which the food item is inspected.
 12. The foodinspection assisting apparatus according to claim 11, wherein the secondcircuitry is further configured to generate an indicating data of aposition of the detected foreign matter or the detected defect, andgenerate the inspection assisting information including an image inwhich the indicating data is arranged at the position of the detectedforeign matter or the detected defect.
 13. A food inspection assistingsystem comprising: the food inspection assisting apparatus according toclaim 11; the conveyer configured to carry the food item on a carryingsurface to the inspection area; and the information output device whichincludes a fourth circuitry configured to display the inspectionassisting information; wherein the third circuitry is further configuredto move a position of displaying the inspection assisting information insynchronization with movement of the food item carried in the inspectionarea.
 14. A food inspection assisting system comprising: the foodinspection assisting apparatus according to claim 11; the conveyerconfigured to carry the food item on a carrying surface to theinspection area; and the information output device which comprises adisplay configured to display the inspection assisting information on ascreen, or a projector configured to project the inspection assistinginformation to a carrying channel in the inspection area or to the fooditem carried in the inspection area.
 15. A food inspection assistingsystem comprising: the food inspection assisting apparatus according toclaim 11; the conveyer configured to carry the food item on a carryingsurface to the inspection area; the information output device configuredto display the inspection assisting information; and a second sensorthat detects a passage of the food item, wherein the third circuitry isfurther configured to perform control to display the inspectionassisting information when the food item is detected by the secondsensor.
 16. A food inspection assisting apparatus comprising: a firstcircuitry configured to acquire a plurality of continuously capturedimages from a first sensor continuously capturing images of a food itembeing carried by a conveyor, and generate a plurality of inspectionassisting information for the food item based on the plurality ofcaptured images; a second circuitry configured to control display timingof the plurality of inspection assisting information to be displayed byan information output device based on time required to convey the fooditem to an inspection area in which the food item is inspected; theconveyer configured to carry the food item on a carrying surface to theinspection area; and the information output device which comprises adisplay that displays the inspection assisting information on a screen,or a projector that projects the inspection assisting information to acarrying channel in the inspection area or to the food item carried inthe inspection area, wherein the second circuitry is further configuredto control to continuously display the plurality of inspection assistinginformation obtained in correspondence with the plurality of capturedimages.
 17. The food inspection assisting apparatus according to claim16, wherein the second circuitry is further configured to move aposition of displaying the inspection assisting information insynchronization with movement of the food item carried in the inspectionarea.
 18. The food inspection assisting apparatus according to claim 16,further comprising a third circuitry that detects a foreign matter or adefect from the captured image of the first sensor, wherein the firstcircuitry is further configured to generate an indicating data of aposition of the detected foreign matter or the detected defect, andgenerate the inspection assisting information including an image inwhich the indicating data is arranged at the position of the detectedforeign matter or the detected defect.
 19. The food inspection assistingapparatus according to claim 16, further comprising a second sensor thatdetects a passage of the food item, wherein the second circuitry isfurther configured to perform control to display the inspectionassisting information when the food item is detected by the secondsensor.